DOCUMENT 



RESUME 



EA 001 977 



j 

! 



ED 026 734 



By* Flanagan, John C. 

Defining Educational Outcomes for Today’s Schools and Assessing Their Attainment. 

Iowa Univ., Iowa City. Iowa Center for Research in School Administration. 

Report No* SR "61 
Pub Date Dec 68 

Note~9p.; Paper prepared for Annual Conference (53rd) on School Administration and Supervision. 

EDRS Price MF'SO.25 HC'$0.55 

Descriptors - Administrator Role, *Educational Improvement, *Educational Objectives, Educational Programs, 
Educational Quality, Educational Research, *Evaluation Methods, ^Student Development, *Student Needs 

A school's educational objective should include three major types of goals for 
each individual: Cl) Preparation for an appropriate occupational role, (2) preparation 
for the responsibilities of citizenship, and (3) preparation for enjoying leisure 
activities. Previous efforts to individualize education have been relatively ineffective. 
An example of current efforts to both broaden and individualize educational 
outcomes is Project PLAN, whose goals include the student's personal formulation of 
goals and plans, with each student given full responsibility for his development. The 
State of Pennsylvania is attempting to collect in a systematic fashion data required 
to define educational outcomes. Two other studies, an American Institutes for 
Research study and Project TALENT, were conducted to develop methods for 
evaluating the attainment of educational outcomes. To improve the quality of their 
educational program, school administrators should make intensive case studies of 
samples of students, including the following items: Community environment, home 
environment, personal history, personal activities, school experience, personal 
characteristics, and personal plans. (HW) 
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DEFINING EDUCATIONAL OUTCOMES FOR TODAY’S SCHOOLS 
AND ASSESSING THEIR ATTAINMENT 

John C. Flanagan 
American Institutes for Research 




There is a growing conviction among educational administrators that American education 
is on the threshold of an era of improved quality in student development. Two very attrac- 
tive resources which seem likely to play a large role in the new educational programs are 
educational technology and the individualization of instruction. However, recent memories 
of the extravagant promises made for educational television, teaching machines, and pro- 
grammed learning, and the deflation following early tryouts of these tools have given the 
administrator greater caution in evaluating proposals which will make possible this big 
forward step. Even though education lacks the types of data which would permit the sound 
application of systems analysis approaches to educational planning and decision-making, 
the recognition of the need for the systematic study of inputs and outputs and a careful 
evaluation of the alternatives is beginning to provide the framework for developing a new 



educational program. 



DEFINING EDUCATIONAL OUTCOMES - A FIRST APPROXIMATION 



Until the early part of this century, the function of elementary education was to give 
students a commakd of the basic skills. The function of the secondary schools was to pre- 
pare students for college. Only a very small percentage of young people were enrolled in 
secondary schools. During the past fifty years as the function of education changed, there 
have been numerous efforts to Mine the desired educational outcomes. Each new committee 
or commission has chosen some specific aspect of educational development as its main 
emphasis. Although the evolution of these statements of educational objectives or goals 
has been slow, it is believed that in recent years there has been general agreement that 
the focus of the schools should be broadened to include three major types of goals for each 
individual. These goals are: (1) to assist him to plan am. prepare for an appropriate 
occupational role; (2) to prepare hin for the responsibilities of citizenship including his 
personal and social development; and (3) to emphasize those aspects of general education 
which will help him find deeply satisfying activities for the anticipated increased time 
available for leisure and recreation. 



Along with this broadening of the definition of educational outcomes, there has been an 

increasing awareness that previous efforts to individualize education have ~ e > n _ re a ve ^ 
ineffective. Homogeneous grouping,, the various track plans, and special enrichment pro- 
grams have not provided each individual student with that education which is most appro- 
priate to his needs and goals . 



An example of current efforts to both broaden and individualize educational outcomes 
is provided in terms of the goals of Project PLAN (A Program for Learning in Accordance 
w ith Needs), The six main goals of this program which are formulated in terms of an 
individualized approach to education are the following: (1 ) Personal formulat ion of g oals an d 
plans. This includes acquainting each student with the extent to which he has developed his 



potential abilities, his interests and preferences, his personal and social behavior 
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patterns, and his skills in making decisions. The successful attainment of this goal also 
reouires the acquisition of knowledge about roles, activities, and opportunities in occupa- 
tions, leisure time activities and social and citizenship roles. On the basis of his lmow- 
ledge of the extent to which his potential abilities have been developed and his knowledge of 
the requirements of various roles and activities, his most important purpose is to formulate 
individual goals and specific plans for attaining these goals. In the past, the educational 
program has primarily told the individual to set high goals for himself and develop high 
aspirations without suitable procedures for informing him of his own unique talents or 
criving him any real information about the requirements for the roles and activities which 
miohfc be open to him. On the basis of the data from the Five Year Follow-up findings from 
Project TALENT, it is now possible to provide each student with a sound basis for making 

his personal decisions. 



Personal development. Thin includes (a) personal adjustment to difficult situations, 
including frustrations and failures; (b) responsibility and effort in carrying out assigned or 
agreed upon tasks; (c) creativity and initiative when faced with an unexpected situation or a 
special need; and (d) integrity with respect to personal behavior, statements, and the pro- 
perty of others. This also has been an area seriously neglected by current educational 
programs. It is proposed to collect observations of specific behaviors in the form of 
critical incidents on these types of development and to use this as a basis for correcting 
ineffective behaviors and reenforcing and encouraging effective hehaviors in this area. 



(3) Social d evelopment. This also includes four main types of behavior: (a) social 
adjustment as shown by helping and sharing with others; (b) sensitivity to others including 
kindness to persons with special handicaps or problems, respecting their race or religion 
though different from one’s own, and seeing that others are not left out; <c) group orien 
including contributions to group activities, good sportsmanship, and similar ac v 
placing group before personal interests; and (d) adaptability to rules and convention mdud- 
tag following rules even when this is unpopular, showing good taste and manners to difficult 
situations, and complying with specific instructions. This also has been a neglected area. 
The program proposes to use the same type of procedures in social development as were 
mentioned in the preceding paragraph on personal development. 



(4) npvplnntng basic skills and abilities. This includes some of the more important 
long-range objectives of education: namely the ability to read with comprehension, the ■ > 
ability to express one’s self, and the ability to think and reason correc y on wr ous typ 
issues. Although this has been the basic function of traditional education, it is 
that an individualized educational program making full use of technologica an au ° 
aids, can greatly improve the efficiency and effectiveness of these developmental Pr°g«ms. 
A very important aspect of developing basic skills and abilities is to commun c ® ® ^ 

to thestudent his current level of ability and information as to the practical implications o 
this level of development in terms of those important tasks which he can carry ou , s«« ess ‘ 
folly and those which will require a greater development of his basic skills and • 

It is proposed that this be done by interpreting his performance on test materials in term 
of their equivalents in these practical, important tasks and activities. 
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(5) Acquiring k nowledge, concepts, principles, and skill in applying these to new 
problems. This includes much of the content of traditional instruction. Emphasis is on the 
concepts and principles and their application rather than on specific items of knowledge 
which are not basic to these larger units of understanding and application. Much of the 
emphasis in traditional education has been on specific items of information rather than con- 
cepts and principles and their application in new situations. The information explosion has 
made it quite impossible for today’s students to learn all of the important items of informa- 
tion in a given area of knowledge. 



(6) Management of individual development. The first of these goals is focused on die 
personal formulation of goals and plans. This last goal focuses on a personal program for 
the management of individual development which is essential for carrying out the varwus 
e d ucatio n al and developmental activities included in the four preceding paragraphs. This 
goal is to make die individual a relatively independent and responsible manager of Ms own 
education and development. Resent educational programs give the student very 1 ™ e 
responsibility in this area. This goal cannot be achieved by merely telling the student he 
has the responsibility and should proceed to manage his development. l£ will require a 
systematic and carefolly monitored program to convey to the student increased .responsibility 
aiming at his taking foil responsibility for his development. The success in achieving this 
goal will be evaluated in terms of the student's demonstrated ability to carry out his pro- 
gram independendy of the teacher and his parents. However, it is not necessary to wait o 
see whether or not he has assumed responsibUity and is effectively managing his program 
for development as a whole. 3oth the initial goal and this last goal can be evaluate y 
developing specific standardized assignments to plan and carry out one particular short-term 

developmental task. 



NEEDED RESEARCH IN DEFINING EDUCATIONAL OUTCOMES 



It has long been recognized that many types of information are essential to d ® fin “S 
educational outcomes. InE. ?. T.indouist's Educational Measurement, published in 1951, 
Ralph W. Tyler suggests that studies of learners, studies of the demands of life out i 
the school, the views of subject matter scholars and philosophers, and studies of the 
psychology of learning are necessary as an appropriate base for formulating educationa 
objectives. Too often, however, no systematic efforts are made to collect these types o 
data. Most statements of educational objectives have been developed by committees and 
commissions who meet over a period of weeks or months and discuss among emse 
what the most appropriate educational outcomes for today’s students are. secon ° 
is that they rarely discuss the problem of the varying needs for individuals having differ 

talents. 



Recently, one effort has been initiated to collect in systematic fashion, some of die 
types of data required to define educational outcomes. In the State of Pennsy vama, 
local school districts, working under the leadership of Raymond Bernabei and Donald Weight 
have undertaken a three year project with substantial support from the Federal Govermnent 
to define, clarify, and improve the goals for quality education in Pennsylvania. One of the 
studies involves the collection of critical incidents from teachers at the fourth, eighth, aim 
twelfth grade levels. These incidents would report observations by the teacher of a spec c 



- 3 - 




i 






I 









R ^li'^^y/^i^-^ w> ^ ■>•. >& ^***'4^*1 



,„i!^.!„.Ui,;J!!U"!BWJ' 






I 



behavior by the student indicating achievement or failure to achieve one of the ten specific 
goals of quality education proposed by a previous study. For example, one of these goals 
is that each child should acquire the greatest possible understanding of himself and an 
appreciation of his worthiness as a member of society. The teachers are being asked for 
specific incidents in which a child clearly indicated that he appreciated or failed to appre- 
ciate his worth as a member of society. From several hundred such incidents t s ieve 
that it will be possible to obtain a more precise definition of this goal of quality education. 



A second type of incident will be obtained in which a teacher behavior led directly to a 
change in student behavior, such as acquiring a skill or changing an attitude or way o 
behaving in a specific situation. An example of this type is a report by the teacher o a 
specific thing that he or she did which produced an observed change in a student s attitude 
towards school and learning as shown by his behavior in a specific educational situation. 



Another type of data collection will involve the intensive study of individual cases with 
special reference to the educational needs of these individuals. At several educationa 
levels and also for a group who have completed their formal schooling, a systema c e 
of the background and development of each of several thousand students will be obtained 

from the students* teachers and parents. 



Using these types of data, it Is proposed that explicit rationales be developed for 
specifying the design of assessment programs for the various goals of quality educa on. 
Such explicit rationales consist of three parts: first, the definition, description, an ^ 
delimitation of each educational objective. These would be based primarily on the critic 1 
behavior categories formulated from the critical incidents. Second, an analysis of this 
behavior in terms of its nature and probable determiners would be undertaken. Third, 
specifications for the types of assessment procedures which will determine whe er o 
the goals have been achieved would be drawn up. 



Supplementing these data with the results from other studies of the conditions of learning 
and the requirements for various roles and activities, it is proposed that committees _of 
teachers, subject matter specialists, scholars, philosophers, and community leaders for - 
ulate a detailed definition of educational outcomes and procedures for assessing these. 



It is believed that the data from the intensive case studies would also provide the basis 
for developing some principles for individualizing education. Many possible eduMtional 
goals are desirable, but for a particular individual with limited time and a specific ability 

to learn, only goals which are most important for achieving the < op “® n 
ess ential for the life goals and plans which he has formulated should be included. It is very 
likely that in many instances greater emphasis on the basic skills such as reading compre- 
hension and effective oral and written expression are of much greater lmpormnce tton th 
learning of many of the items in our cultural heritage. The effort of the Pennsylvania suidy 
is to develop a comprehensive set of educational objectives which include approp 
lects for each of our young people. The changing society and technological and scientific 
advances together with the expected increase in understanding of human iearning processes 
will undoubtedly require that these objectives be in a constant state of revision and modifl 
cation. However, tee time is very much overdue to make a substantial systematic effort 
collect tee necessary data to define a broad program of educational outcomes to be made 
available by American schools. 
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APPROACHES TO EVALUATING THE ATTACHMENT 0? EDUCATIONAL OUTCOMES 

* 

In a recent study conducted by the American Institutes for Research in a small Eastern 
city, a number of new procedures were used in evaluating the educational development of 
individual students. In each of Grades 4, 6, 9, and 12 a 10% sample of the students was 
selected in such a way as to be representative of the total group of students for each of these 
classes in terms of learning ability. For the students in die sample, various types of data 
were collected, These included the pertinent information in the cumulative record for each 
of the students; information regarding their school program which was collected by ques- 
tionnaire for Grades 6, , and 12 students and by interview from die Grade 4 students; 
consultations with teachers and specialists regarding some of die students* problems; and 
an interview with each student by the evaluator. 



On the basis of all the data obtained, the evaluators answered several questions regard- 
ing the educational program. One of these questions was, "What is the quality of student 
motivation for learning?" For the four classes combined the evaluators rating of the 
motivation for learning was outstanding or excellent for about 30% of the students and fair 
or good for 60% and poor or very poor for 10%. It is interesting to note that there was a 
definit e downward trend in motivation on the part of the students in the judgment of these 
evaluators. For example, in the fourth grade, 41% of the students were rated as having 
excellent or outstanding motivation, while in the twelfth grade, only 12% of the students were 
rated as having excellent or outstanding motivation for learning. Other evidence suggests 
that this finding would be duplicated in most schools throughout the country. 

The evaluators were also asked to indicate the overall effectiveness of the school in 
meeting the needs of children in education and guidance. The evaluators reported that the 
schools were meeting the needs of the students "nearly perfectly" for 25% of the students, 

"in most respects" for 49% of the students, "well in some respects and poorly in others" 
for 22% of the students, and either "fairly unsatisfactory" or "very poorly" for 4% of the 
students. The unsatisfactory evaluations were given for such reasons as schools having 
failed to detect a severe visual handicap, misplacement of the student, failure to provide 
psychological assistance, lade of remedial instruction in the early grades, and lade of 
personal counseling. 

Although student evaluations are questionable in some respects, it seemed worthwhile 
to ask these students whether the instruction in a particular dass made then want to learn 
more or not want to learn more about the subject than was required in the course. In re- 
plying to this question at the ninth grade level, the only subject in which close to half the 
students felt the instruction made them want to learn more was science. The subject making 
the poorest showing in thi s respect was English where less than one in five indicated that 
they were motivated to Jeam more than was required. 

In addition to studying a sample of the students in schools, it was believed that useful 
evaluative information could be obtained from recent graduates. All of the students who had 
graduated in the class of i; 60 and 1964 in the high school in this system were sent ques- 
tionnaires in the summer of 1 65. With the aid of reminders and further follow-ups over 
the next six weeks, 84% of the 753 members of the class of 1 >64, and 72% of the 429 grad- 
uates in the class of 1960 returned complete questionnaires. These high rates of return, 
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plus the consistency of the findings in the two classes indicate considerable reliance can be 
placed on the findings reported. One of the items on the questionnaire asked students to 

complete the sentence, "The best thing provided for me by this high school was »" 

In both classes the most frequent responses were, "a good basic education, " and "good 
preparation for college." In both classes another frequent response was "good competent 
interested teachers." 

Another item in the questionnaire asked that they complete the sentence, "The main 

thing I believe I needed which was not provided by this high school was c " 

Approximately 25% of the students in each class indicated that "guidance and counseling" 
was the most important u nfil led need in their high school education. The only other item 
which was indicated as a need by an appreciable proportion of the students in the two classes 
was "assistance in learning how to study," The other replies were scattered over a wide 
variety of needs. 

These graduates were asked to provide an overall evaluation of the preparation they 
received in the schools by answering the following question, "Immediately following gradu- 
ation from hign school, how well do you believe you were prepared to enter into your new 
role as a paid worker or student':" About 30% of each of the classes indicated that they were 
very well prepared; 45% in each class indicated they were fairly well prepared; the rema in i n g 
25% in the two classes indicated they were generally prepared, but lacked some specific 
or they were not well prepared. 

On the basis of a systematic study of all available data from the students and observa- 
tions and studies of the curriculum and staff, it was recommended that the greatest improve- 
ment in the school program could be expected from focusing on the two concepts of individu- 
alization of education and formulation of realistic goals and plans by each student to prepare 
himself for important life roles. 

Some of the data from Project TALENT, a survey of more than 400, 000 secondary 
school students conducted in 1 "60, also provide evaluative data for this country’s educational 
program. An indication of their lade of knowledge with respect to themselves and various 
career opportunities is shown by the feet that the career plans reported by boys one year out 
of high school remained the same as those they expressed in the tenth grade for only 19% of 
them and were the same as those given a year earlier in the twelfth grade for only 31%. 

For the girls the numbers having the same plans one year after graduation were 29% when 
compared with Grade 10 plans and 41% for the Grade 12 plans. 

Evidence of the ineffe '.tiveness of much of the current instructional material was also 
provided by Project TALENT. The students were asked to answer the question, "How often 
do you read material over and over again without really understanding what you have read / 
Sixteen percent of the boys and 17% of the girls in the twelfth grade reported that this hap- 
pened to them most of the time. In this twelfth grade group, an additional 18% of both the 
boys and the girls reported that this happened about half the time. Thus we have the clear 
indication that for more than a third of our boys and girls in the twelfth grade, the textbooks 
are so difficult that half or more of the time they are unable to understand what they read. 

Or conversely, their instruction in understanding what they read has been deficient in pre- 
paring them for reading materials appropriate to this level. 
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Although the methods of evaluating the attainment of educational outcomes are definitely 
in need of improvement, it is clear that information is already available to indicate a 
number of the deficiencies of current educational programs and suggest ways of improving 
them. 



SUGGESTIONS FOR SCHOOL ADMINISTRATORS AND SUPE 3. VISORS 



It is proposed that the best type of study for the school administrator and supervisor to 
undertake to provide a basis for improving the quality of their educational program is to 
make intensive case studies of a sample of students. These studies should be focused on the 
educational development of the student and the appropriateness of "he educational program for 
assisting him in this development. Such an intensive study of individual needs should include 
the following types of items: 



(1) Community environment. This would include something about the adult occupa- 
tions in the community, the level of education of the adults in the community, the type and 
quality of housing, the recreational opportunities, social opportunities, library and cultural 
opportunities, work opportunities especially part-time and summer opportunities for student 
work, and the size and the type of the community. 



(2) Home enviro nment. This would include such items as living quarters, their type 
and quality, the father's and mother's occupation, the parent’s education, the family 
recreational activities, the family cultural opportunities, the family chores done by the 
children, the personal living and developmental facilities available to the student, the family 
transportation means, the appliances both for housework and recreation, the availability of 
work shop and tools, the distance of the school from the home, and the transportation to the 
school, the number and ages and characteristics of siblings. This should also include the 
parent's views about the child's abilities, their expectations for the child, and their plans 
for assisting the child to obtain advanced education or training. 



(3) Personal history. This should include such items as moves, age at school en- 
trance, grades repeated or skipped, school marks, special services from the schools, 
health history, test scores, teacher comments, special developmental activities such as 
music lessons, work history, personal achievements and recognitions, personal behavior 
problems, and social problems. 



(4) Personal activities. This would include such items as the hours per week spent 
in study, work, sports, watching TV, listening to radio or recordings, attending movies, 
social activities, specific hobbies, and recreational reading. 



(5) S chool expe rience. This would include such items as his interest in his instruc- 
tional materials in various courses, his motivation to learn in various subjects, his views 
of the value for him of classroom activities, his views concerning the progress he is making 
in his own development, his views concerning the worth of the learning activities and goals 
set by the teacher and the school, his evaluation of special aspects of the school program, 
and an overall evaluation of his opportunities for educational development. 
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(6) Personal characteristics . This would include the student's reading level, interests 
study skills, and special abilities, 

(7) Personal plans. This would include the student's plans with respect to education, 
career, military service, marriage, children, cultural and leisure time activities, civic 
responsibilities, and general life goals and values , 

Enough case studies are needed to provide a sound basis for modifying and revising the 
school program. Probably at least one hundred students at a grade level are needed. On 
the basis of these data, it is believed that the administrator and supervisor can select and 
devise an educational program which will fit each individual much more appropriately than 
do current programs, This type of exercise is believed to be the most effective method 
of bringing to the attention of the school authorities and teachers, the major deficiences of 
the present educational program. It should also be very valuable in emphasizing the 
strengths of the current program so that these are not lost in the efforts to Improve the 
program for certain students. 

To improve the quality of educational outcomes a definition of the specific needs of 
each student is required. These are ever-changing and evolving but decisions must be 
made now for today's students. 



